In order to monitor extended areas, or to acquire reconnaissance data, a video imaging device (hereinafter—video camera) may be mounted on an elevated platform. The platform may be a piloted aircraft, an unmanned aerial vehicle (UAV), a free-flying dirigible, a tethered balloon, a tower, or any other elevated platform on which a video camera may be mounted. When topographical or other conditions permit, the platform may also be a manned or unmanned ground vehicle, boat, or any other type of platform that is not elevated. The video camera may be any imaging device capable of providing a continuous or truncated video feed, such as, for example, a video camera recording a series of successive frames, or a video camera recording individual frames at intervals. The video camera may acquire images formed by visible light, infrared radiation, or by radiation in any other spectral range. The video camera may transmit the acquired image data to a remote, or local, receiver and display.
At any given moment, the video camera may acquire an image of a scene within the monitored area that falls within the field of view (FOV) of the video camera. The video camera is described as pointing at that scene. The video camera FOV is limited by characteristics of the video camera structure and its optics. In order to acquire an image of a different scene within the monitored area, the video camera is reoriented so as to point at that different scene. The different scene then falls within the video camera FOV. For this reason, the video camera is mounted in such a manner as to allow its orientation in any of a wide range of directions. Often the orientation of a video camera is defined by its pan and tilt angles, or by the direction of its line of sight (LOS).
A video camera operator at a remote location operates the video camera by means of a remote-control console. Alternatively, the video camera may be operated by an automated system. The video camera operator also views the video camera image output on a display associated with the remote-control console. In general, the video camera operator points the video camera at various scenes that require monitoring. Examples of such regions include roads, borders, areas surrounding places of business, forested areas and other areas of interest. The video camera operator views video camera output images to monitor those scenes. While monitoring the scene, the video camera operator may identify a feature in an output image of the scene that requires further attention. The video camera operator may also be able to distinguish between areas where a certain type of feature would require further attention, and other areas where such a feature would not raise interest. For example, vehicle traffic close to a border or in the parking lot of a business that is closed for the night may require further attention, whereas vehicle traffic on an intercity road may not. A vehicle traveling at high speed on a road with a maximum speed limit may require further attention, while a vehicle traveling at similar speed on a road with a different speed limit may not. Similarly, a fire in a forested area or covering a wide area may require further attention, while a fire covering a small area in a designated campground may not. Further attention to a feature may include further examination of that feature via the remote video camera by pointing the video camera at that feature for an extended period of time, or sending an observer to observe the feature from close range.
In general, operating a video surveillance system is a tiresome and eye-straining task. When operating a video camera, a human operator is constantly looking for objects of interest, and must gaze at the monitor for a lengthy period of time. The operator often must search for features with in the display to verify that the video camera is indeed aimed at the intended scene or region of interest. In general, in the absence of obvious and unambiguous visual cues, such verification may be tedious and time-consuming.
When a monitored area is substantially larger than the FOV of the video camera, the operator must also redirect and point the LOS of the video camera at various locations within the monitored area. These locations may cover the entire monitored area or may include separate isolated regions of interest.
Alternatively, the video camera pointing may be controlled by an automated system. Automated video camera pointing systems are generally based on simple, repetitive motions. For example, in a pushbroom system, the video camera is pointed in a fixed direction while the motion of the platform causes the field of view (FOV) of the video camera to scan across the monitored area parallel to the direction of motion of the platform. In a sweeping system, the video camera is caused to periodically rotate sideways, sweeping laterally across the direction of motion of the platform. With such automated systems, the coverage of the monitored area by the video camera FOV is not related to the necessity or interest in covering the area. Thus, use of such systems may result in inefficient use of time, by spending much of the time monitoring regions of little or no interest.
Continued monitoring of similar or repeated scenes tends to be a tedious and monotonous task for a human operator. In general, features that require further attention are relatively uncommon, so that monitoring of the scenes may fail to hold the operator's attention after a period of time. The operator may become fatigued and less alert and, as a result, may fail to point the video camera at a scene that requires monitoring. A fatigued operator may fail to identify a feature requiring further attention. Furthermore, in images of complex scenes, or of scenes that are viewed intermittently, an operator may have difficulty detecting slowly or evolving or instantaneous changes or small changes that might require further attention. Furthermore, as scanning tasks typically include monitoring of several, dislocated, scenes, the operator is also required to manage the scanning tasks and redirect the LOS of the video camera from scene to scene in order to monitor all the required locations following a pre-defined re-visiting schedule. As the operator fatigue is increased, his ability to monitor all the different areas in an efficient manner substantially decreases over time.
Therefore, there is a need for a system that automates the remote monitoring process, automatically and efficiently pointing the LOS of the video camera at regions of interest, assisting the operator in finding and detecting those features in the monitored areas that require further attention.
It is an object of the present invention to provide an automatic surveillance system and method that may assist a video camera operator by automating the pointing of a video camera that is mounted on a UAV or other platform. It is a further object of the present invention to assist the operator by automating the detection of features in the video camera output that require further attention.
Other objects and advantages of the present invention will become apparent after reading the present specification and reviewing the accompanying drawings.